1. Field of the Invention
The present invention relates to an imaging device, an imaging method and a computer program for capturing an image containing an image of a person as a subject. In particular, the present invention relates to an imaging device, an imaging method and a computer program for performing automatic camera work in response to recognition of a particular pattern from a subject represented in the image.
The present invention further relates to an imaging device, an imaging method and a computer program for automatically determining a photographing timing in response to a recognition of a smiling face of the subject contained in the image. In particular, the present invention relates to an imaging device, an imaging method and a computer program for determining a photographing timing by detecting a smiling face of an appropriate subject with an input image showing a plurality of subjects.
2. Description of the Related Art
Cameras have a long history serving as means for recording visual information. Currently, digital cameras are replacing silver-salt cameras using a photographing plate or film and are in widespread use. The digital camera captures an image with a solid-state imaging element such as a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) device and analog-to-digital converts the image. The digital camera stores a digital encoded image onto a memory and performs an image processing operation and an image management on the image with a computer. Such a digital camera is free from the limitation of service life of films.
The silver salt camera and the digital camera are substantially identical to each other in basic camera work. The digital camera incorporates advanced automatic technique in camera work. Even an inexperienced user is unlikely to fail to photograph.
The digital camera has an auto focus (AF) function, an auto exposure (AE) function, and an image stabilization function. The digital camera has an advantage that the AF function and the AE function are performed not only by an optical system but also in digital processing. Furthermore, the digital camera performs appropriate color reproduction by image quality adjusting a captured image in digital processing. The digital processing includes multiplying the captured image by a white balance gain in an auto white balance function.
The auto focus and the image quality adjustment are performed with the camera focusing on at least one subject present within an image frame. Previously, the user manually sets an important subject. Currently, the digital camera incorporates photographing techniques of face detection. For example, the camera detects a face of a person from a captured image and automatically focuses on the detected face or automatically adjusts image quality with respect to the face.
Japanese Unexamined Patent Application Publication No. 2003-271933 discloses a technique of detecting a face from a captured image signal. In accordance with the disclosure, an average face image serves as a template to be checked against an input image signal for matching. Such a technique is widely known. A face slant in the captured image is difficult to detect. In such a case, an angle of the face is detected, the image of the face is rotated in accordance with the detected angle value and then the matching process is performed. Japanese Unexamined Patent Application Publication No. 9-171560 discloses a technique of detecting an angle of a face. In accordance with disclosure, the centerline of the face is detected and the likeliness of the centerline is calculated based on left-right symmetry of the face. The angle of the centerline having a high likeliness is output as the angle of the face.
Auto shutter for determining a photograph timing based on a recognition of a particular pattern of a subject image such as a smiling face has been recently studied and developed.
Japanese Unexamined Patent Application Publication Nos. 2000-347278, 2004-294498 and 2006-237803 have disclosed cameras that output a photographing timing signal when a predetermined photographing condition is satisfied. Such a camera determines whether a main subject satisfies the predetermined photographing condition within an area of interest contained in the main subject to be photographed. The areas of interest may include a shape and color of the eyes and the mouth of a person if the person is photographed and feather of a bird if the bird is photographed.
Factors serving as specific photographing timing may be conditions to make a person look good in photo, such as “remaining unblinking,” “looking at the camera,” and “smiling.” In related art, retouching techniques have been long available to modify a photo of a person to make the person look good. However, it is difficult to modify a photo of a person blinking to a photo of the person remaining unblinking or to modify a photo of a person with a smiling face to a photo of the person with an unsmiling face. This is one reason why a smiling face is one photographing condition.
The photographing condition of “smiling” is determined based on image information related to an area of the white of the eyes in the subject, the horizontal length of the mouth of the subject, and an area of white portions in the mouth. If a plurality of subjects are shown in an image frame input via an imaging element, it is not known which one of the subjects is to be used to determine the photographing timing.
Japanese Unexamined Patent Application Publication No. 2006-33437 discloses a digital camera that detects a plurality of faces. In accordance with the disclosure, an outline is set up on the largest face or the closest face in an AF area. When face detection is performed using a through image, the through image is increased in luminance to facilitate face detection to a level higher than luminance used when no face detection is performed. The digital camera does not perform an automatic photographing operation based on smiling face detection. The detected face set within the AF area is automatically determined by the camera. The user cannot specify a subject having a smiling face the user is interested in.
Japanese Unexamined Patent Application Publication No. 2006-145629 discloses an imaging device. In accordance with the disclosure, the imaging device performs a focus adjustment so that a focusing lens focuses on a face closest to the imaging device from among detected faces. An outline surrounding the focused face is displayed on a liquid-crystal display. If the face surrounded by the outline is not a desired person's face, the user may change the outline onto another face using cross keys. The imaging device performs auto focus process by determining a main subject with respect to the face closest to the imaging device, but does not performs the auto focus process based on smiling face detection. The imaging device allows the user to specify a subject desired by him or her, but the user can specify only a particular subject.
The auto focus process is performed, basically focusing on a single point although the manner of focusing depends on characteristics of an optical system, such as focus depth. The imaging device cannot focus on a plurality of subjects at a time. The auto focus process may be performed based on the smiling face detection. For example, persons in an event such as a party may be photographed in a relatively large viewing angle. In such a case, the photographing condition may be that all members in the party are smiling instead of any particular single person smiling. The automatic photographing operation based on the smiling face detection is substantially different from the automatic photographing operation based on auto focusing.